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AN  EXPONENTIAL  AUTOREGRESSIVE-MOVING  AVERAGE  PROCESS  EARMA(p,q); 
DEFINITION  AND  CORRELATIONAL  PROPERTIES 


ABSTRACT 


A new  model  for  pth-order  autoregressive  processes  with 
exponential  marginal  distributions  EAR{p)  is  developed  and  an 
earlier  model  for  first  order  moving  average  exponential  processes 
is  extended  to  qth-order,  giving  an  EMA(q)  process.  The  correla- 
tion structure  of  both  processes  are  obtained  separately.  A 
mixed  process,  EARMA(p,q),  incorporating  aspects  of  both  EAR(p) 
and  EMA(q)  correlation  structures  is  then  developed.  The 
EARMA(p,q)  process  is  an  analog  of  the  standard  ARMA(p,q)  time 
series  models  for  Gaussian  processes  and  is  generated  from  a 
single  sequence  of  independent  and  identically  distribution 
exponential  variables . 
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1 . INTRODUCTION 

The  first-order  autoregressive  sequence,  EAR(l),  was 
introduced  by  Gaver  and  Lewis  (1975-1978)  with  the  primary  aim 
of  generalizing  the  Poisson  model  for  point  processes  to  one  in 
which  the  intervals  between  events  were  correlated  but  still  had, 
marginally,  exponentially  distributed  intervals.  The  EAR(l) 
sequence  is  a simple  random  linear  combination  of  independent 
exponential  random  variables  whose  properties  are  relatively 
simple  to  derive.  This  is  in  contrast  to  previous  attempts  to 
generalize  the  Poisson  process  via  Markov  dependence  which  led 
to  intractable  models  (see  e.g.  Wold,  1948  and  Cox,  1955). 

In  Lawrance  and  Lewis  (1977)  another  sequence  of  dependent 
exponential  random  variables  wab  introduced.  This  sequence, 
called  ENA(l),  was  again  a random  linear  combination  of  independent 
exponential  random  variables,  but  had  the  dependency  properties 
of  a first  order  moving  average  process.  The  first-order  moving 
average  and  autoregressive  processes  were  combined  by  Jacobs 
and  Lewis  (1977)  to  form  the  EARMA(1,1)  sequence.  Jacobs  and 
Lewis  (1977)  gave  stationary  initial  conditions  and  mixing 
properties  of  the  sequences,  these  results  applying  to  the  EAR(l) 
and  EMA(l)  processes  as  special  cases. 

In  the  present  paper  we  extend  these  results  and  describe 
a mixed  pth-order  autoregressive,  qth-order  moving  average  process 
with  exponential  marginal  distributions  which  we  denote  as 
EARMA(p,q) . The  process  is  again  a random  linear  combination 
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of  independent  exponential  variables,  and  as  such  is  simple  to 
generate  on  a computer;  it  will  thus  be  useful,  for  example,  in 
simulation  studies  of  queues  with  correlated  Interarrival  times 
or  service  times  (see  Jacobs,  1978).  The  process  is  not  unique,  , 

but  its  correlation  sequence  {p  } does  satisfy  equations  like 

iv 

the  Yule-Walker  equations  which  arise  in  the  study  of  linear 
processes  (see  e.g.  Feller,  1966,  or  Box  and  Jenkins  1970). 

It  is  perhaps  well  to  reiterate  the  essential  difference 
between  the  EARMA(p,q)  process  and  the  ARMA(p,q)  process;  this 
is  that  the  EARMA(p,q)  process  is  defined  to  have  an  exponential 
marginal  distribution.  It  is  not  known  how  one  would  pick  the 
error  sequence  in  the  ARMA(p,q)  sequence  to  make  it  have,  even 
approximately,  marginal  exponential  distributions.  In  fact  the 
marginal  distributions  would  tend  to  be  approximately  normally 
distributed  for  most  error  sequences  (Mallows,  1967);  the  catch 
in  this  result  is  that  the  distribution  of  the  error  sequence  be 
independent  of  the  parameters  of  the  moving  average  and  auto- 
regression. This  is  not  so  for  the  EARMA(p,q)  process. 

This  paper  will  be  limited  to  definitions  and  to  description 
of  the  correlational  properties  of  the  EARMA(p,q)  process.  In 
Section  2 the  EAR(p)  model  is  introduced  and  an  explicit  solution 
for  the  required  error  process  for  autoregression  of  order  2 is 
given.  In  Section  3 we  describe ’the  extensions  of  the  EMA(l) 
model  to  the  EMA(q)  model;  these  are  relatively  straightforward  • 

and  are  indicated  in  Lawrance  and  Lewis  (1977)  . The  general 

. 
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EARMA(p,q)  model  is  introduced  in  Section  4,  and  specific  results 
are  obtained  for  the  EARMA(1,1),  £ARMA.(2,1)  and  EARMA(1,2)  models  in 
Section  5 . 

In  deriving  correlational  properties  it  is  assumed  that 
the  EARMA(p,q)  process  is  stationary.  The  question  of  stationarity , 
stationary  initial  conditions  and  mixing  properties  will  be  con- 
sidered elsewhere,  as  will  be  questions  of  distributions  of  sums 
of  the  dependent  variables  and  spectra  of  point  processes  with 
EARMA(p,q)  interval  structure.  There  are  also  open  questions  of 
estimation  of  parameters  and  fitting  to  data. 

We  note  too  that  there  is  a mild  degeneracy  to  the  EAR(p) 
process  in  that  one  obtains  runs  in  which  the  variables  are 
scaled  versions  of  the  previous  variables.  This  disappears  when 
the  moving  average  component  is  introduced.  Another  drawback  is 
that,  unlike  the  ARMA(p,q)  model,  only  positive  valued  serial 
correlations  can  be  obtained  from  the  EARMA(p,q)  model  and  while 
much  data  appears  to  be  of  this  type  (see  e.g.  Lewis  and  Shedler, 
1977),  it  is  a drawback.  This  can  be  overcome  by  considering 
antithetic  processes  but  this  aspect  of  the  model  is  beyond  the 
scope  of  the  present  paper. 
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2.  THE  EXPONENTIAL  AUTOREGRESSIVE  EAR(p)  MODEL 
2a . Definition. 

The  standard  linear,  first-order  autoregressive  model 
for  a stationary  sequence  of  random  variables  {X^}  is  defined 
by  the  equation 


Xi  = P^i-i  + ' 


i = 0,+l,+2,  •••  , 


(2.1) 


where  p is  a constant  which  is  less  than  1 in  absolute  value 
and  {e^}  is  a sequence  of  independent  and  identically  distributed 
random  variables.  Gaver  and  Lewis  (1975-1978)  showed  that  if 
the  {X^}  sequence  were  to  have  an  exponential  marginal  distri- 
bution with  parameter  A,  then  the  parcuneter  p should  be  greater 
than  or  equal  to  zero  and  less  than  one,  and  should  be  zero 

with  probability  p and  an  exponential (A)  random  variable,  E^, 
with  probability  1-p . Thus 


i-1  ^i 

i = 0 , +1 , +2 , 

pXi-i 

w.p. 

P / 

i = 0 , +1 , +2 , 

pXi-i  t E. 

w.p. 

1-P. 

where  {E^}  is  an  i.i.d.  sequence  of  exponential (A)  random 
variables.  Note  that  for  this  EAR(l)  model  the  distribution  of 
the  depends  on  p,  the  multiplicative  weight  of 
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This  violates  an  assumption  which  is  implicit  in  many  applica- 
tions of  (2.1),  the  so-called  AR(1)  model.  In  particular  standard 
results  showing  that  the  sequence  becomes  a normal  process 

as  p ■+■  1 for  any  {e^}  sequence  are  invalid;  in  the  EAR(l) 
process  the  X^'s  always  have,  by  construction,  an  exponential ( A ) 
marginal  distribution. 

Generalization  of  the  usual  higher-order  autoregressive 
models  AR(p)  based  on  extensions  of  (2.1)  to  higher  order 
autoregressive  exponential  processes  is  difficult.  This  is 
because  it  is  not  possible  to  solve  the  defining  equation  for  the 
distribution  of  the  e^'s,  if  it  exists.  We  present  here  a 
different  type  of  p-th  order  autoregressive  models  with 
exponential  marginal  distributions.  They  share  with  the  AR(p) 
models  the  same  correlation  structure,  are  p-th  order  Markov 
processes,  and  are  (autoregressively)  functions  of  at  least  one 
of  the  previous  p variables. 

The  second-order  model,  EAR (2) , takr^  the  form 


w.p. 

w.p. 


(2.3) 


where  and  are  constants  (0  < Oj  < 1)  and  we  show 

later  that  the  distribution  of  the  is  uniquely  determined 

by  the  requirement  that  the  X^'s  have  exponential  marginal 
distributions.  The  second-order  autoregressive  nature  of  the 
model  is  evident;  X^  is  always  a function  of  one  of 
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This  is  in  contrast 


the  previous  two  values  and 

to  the  AR(2)  model  in  which  is  a function  of  a linear  com- 

bination of  X.  , and  X. 

1 ^ 4>« 

The  third -order  model  is  given  by 


X. = ! 

“i^i-i 

w.p. 

M 

1 

Q 

to 

“2^i-2 

w.p. 

02  (l“a3) 

+ Ei  . 

(2.4) 

“3^i-3 

w.p. 

“2“3 

where 


The  mixing  probabilities  and  the  weights  on  the  auto- 
regressed  variables  are  to  some  extent  a matter  of  choice  (other 
parametrizations  are  clearly  possible)  and  we  have  been  guided 
by  two  considerations;  having  a minimum  number  of  parameters. 


preferably  the  same  number  of  parameters  as  the  order  of  the 

autoregression,  and  by  the  need  for  the  autoregression  in  the  EAR(p) 

model  to  reduce  in  order  by  one  when  the  last  coefficient,  a , 

P 

is  set  to  zero.  In  the  present  parametrization  this  implies  the 
weak  restriction  that  it  is  not  possible  to  suppress  intermediate 
autoregressions,  i.e.  dependence  of  on  ^ EARMA(2) 

model . 

With  regard  to  the  question  of  parametrization,  one  could 
in  (2.3)  replace  the  probabilities  aj^=  (1-02)  ^2  “ “2  (1~P) 

and  p and  there  is  then  no  need  for  the  weights  and  02 

to  be  less  than  or  equal  to  one.  But  if  they  are  not,  the  process 
will  not  reach  equilibrium  unless  p is  suitably  chosen,  i.e. 
be  stable.  Again,  even  if  the  process  is  stable  it  is  not  clear 
yet  that  the  additional  parameter  adds  any  generality  to  the 
process.  We  consequently  consider  only  the  parametrizations 
given  in  (2.6). 


2b . The  error  sequence  { e ^ } 

We  now  obtain  the  distribution  of  the  i.i.d. 

sequence  which  will  ensure  that  the  {X^}  sequence  in  the  EAR (2) 

model  has  an  exponential  marginal  distribution.  Let  (f)  (s) 

and  (s)  be  the  Laplace-Stielt jes  transforms  of  the  marginal 

i 

distributions  of  the  X^ ' s and  the  e^^'s: 

-X.s  -e . s 

.t>  is)  = E(e  ^ ) ; 4,  (s)  = E(e  ^ ) . (2.7) 

Xi  H 
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Then  from  Equation  (2.3)  we  have 


(J>  (s)  = [(l-a.)  (p  (a,s)  + (a,s)  ) <|)^(s)  , (2.8) 

^ ^i-1  ^ ^ ^i-2  ^ ^ 

where  we  have  used  the  fact  that  expectation  of  the  mixture  of 
two  dependent  random  variables  is  the  mixture  of  the  expectations 
of  the  marginal  random  variables,  here  and  Xi_2*  Thus 

we  avoid  the  joint  Laplace-Stietl jes  transform  which  comes  in 
when  one  tries  to  solve  the  usual  linear  AR(2)  equations  to 
obtain  an  exponential  process.  Assuming  marginal  stationarity 
for  the  process,  we  have 


<t)x(s)  = [(1-012)  (})jj(aj^s)  + a24'x(“2®^^  * (2.9) 


To  show  that  such  an  error  sequence  {e^}  exists  we  solve  (2.9) 
directly  and  invert  the  transform.  We  have  for  the  EAR  (2)  model, 
using  the  key  requirement  that  the  marginal  distribution  of  the 
X.'s  be  exponential  (A)  , and  thus  (})y(s)  = A/(X+s),  that 


(t>x(s) 

” (1-02)  (t)jj(aj^s)  + a2<{>jj(°‘2®^ 


(2.10) 


( A+aj^s)  (A+a2S) 

TX+iTTTI-cr^TTX+a^¥l  + 02  ( A+a^^s)  ] 


(2.11) 


Then  by  a partial  fraction  expansion 


t 


0^(5) 


A 

A+s 


+ 


A 

A+Ss 


where  S = (l+aj^-a2)  a2 . Using  the  fact  that  and  02  are 

probabilities,  it  is  easily  verified  that  tTq,  and  are 

positive,  and  since  their  sum  is  equal  to  1,  we  have 
0 £ tTq,  tt^,  tt2  £ 1.  Thus  e is  a convex  mixture  of  a discrete 
component  and  two  exponentials,  and  thus  has  a proper  distribution. 
This  distribution  is  also  unique,  by  the  unicity  theorem  for 
Laplace-Stielt jes  transforms.  The  complete  specification  of 
is,  for  i = 0,  + 1,  + 2,  ... 


1 0 w.p. 

aj^/{l+aj^-a2}  , 

e . = 

1 

( E^  w.p. 

(l-Uj^)  (1-02) / [1-02  {l+a2^-oi2)  ] ' 

(2.12) 

1 E^/S  w.p. 

(1-02)  (0^-02)  V[  (l+a^-a2)  (1-S)]  , 

where  {E^}  is  again  an  i.i.d.  sequence  of  exponential  (A) 
random  variables.  It  is  obvious  from  (2.12)  that  the  mean  and 
variance  of  depend  on  and  a^,  the  multiplicative 

weights  of  and  Xj^_2  respectively,  as  well  as  on  A. 

As  in  the  EAR(l)  model  there  is  a non-zero  probability 
of  being  zero;  otherwise  it  is  E^  or  a scaled  version  of 

E^ . The  higher  order  models  can  in  principle  be  similarly 
treated,  although  above  the  third  order  there  will  be  difficulty 
with  the  partial  fraction  expansion. 
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2c.  Correlation  structure 


The  correlation  structure  of  the  stationary  EAR(p) 
models  can  be  obtained  by  the  usual  device  of  multiplying  the 
defining  equations  for  by  for  r = 1,2,...,  and 

taking  expectations.  What  results  are  difference  equations 
which  are  entirely  analogous  to  the  Yule-Walker  type  equations 
obtained  for  the  standard  AR(p)  model. 

Thus  taking  the  EAR (2)  case  as  a typical  example,  we 
have  from  (2.3)  that 

E(X^Xi_r)  = (l-a2)  [aj^E(X^_j^X^_j.)  +E(X^_j.)  E(e^)] 

+ a2[a2E(X^_2X^_j.)  + E(X^_^)  E(e^)]  . (2.13) 

-1  -2 

Using  the  fact  that  E(X^)  = X , var(X^)  = X , because  the 
process  has  an  exponential  marginal  distribution,  and  from  (2.3) 
that 

E(e)  = (1-02)  (l-Uj^+a^)  ' (2.14) 

we  obtain  for  the  correlations  = Corr  (X^,Xj^_^)  the  equation, 

Pj.  = “l  ^r-1  °‘2‘^r-2  (2.15) 

with  = p_^  and  pQ  = 1.  For  the  general  EAR(p)  process 

there  is  the  corrasponding  equation 


Pr  = “i^iPr-i  + “2®2Pr-2  + ***  + Vp^r-p  > D • <2.16) 


Equation  (2.15)  is  a system  of  second  order  difference 
equations  from  which  we  can  obtain  the  following  results. 

(i)  The  solution  of  the  difference  equation  (2.15)  is  (see, 
for  exeunple.  Box  and  Jenkins,  1970,  pp.  58-59) 


(r  > 1) 


Zj^(l-Z2)z^  - Z2(l-z^)z| 


[Z1-Z2] 


L+z^Z2] 


(2.17) 


(2.18) 


where  and  Z2  are  reciprocals  of  the  roots  of  the 


characteristic  equation 


1 - (1-02)3  - 02®  = 0 , 


2 2 2 

and  the  roots  are  real  since  aj^(l-a2)  + 4a2  0 . Also 

0 £ Z2  < Zj^  < 1.  An  implication  of  these  results  is  that 
the  serial  correlations  are  positive  and  eventually  decay 
geometrically,  i.e.  like  have  assumed  that  02  > 0; 

otherwise  we  have  the  EAR(l)  model. 

(ii)  The  correlations  and  P2  can  be  uniquely  defined  in 

terms  of  the  parameters  and  02,  and  vice  versa;  this 

follows  from  (2.15)  for  r = 1 and  r = 2.  We  have 
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w 


(2.19) 


(2.20) 


if  a2  ^ 0.  If  02  = 0 the  model  reduces  to  the  EAR(l) 
model  of  Gaver  and  Lewis  (1975-1978),  and  . 

Equation  (2.20)  may  be  used  to  obtain  Yule-Wal)cer  estimates 
of  and  02  from  estimates  of  the  first  two  serial 

correlations . 

2 

(iii)  If  02  ^ 0 then,  unlike  the  EAR(l)  case  in  which  P2  = Pj^» 

2 

we  have  p2  > P^.  This  can  be  seen  from  (2.19),  which  can 
be  written  as  + o^d-P^)  >.  P^ . Note  too 

that  there  are  values  of  Oj^  and  O2  for  which  P2  > pj^; 
thus  the  additional  degree  of  autoregression  produces,  at 
least  for  the  first  two  serial  correlations,  a broader 
correlation  structure  than  is  possible  with  the  EAR(l) 
process  (02  = 0) . 

(iv)  One  way  to  measure  the  amount  of  correlation  which  is  intro- 
duced into  the  sequence  {£^1  by  the  autoregression  is  by 
an  index  of  dispersion  (Cox  and  Lewis,  1966,  p.  71).  This 
is  just  the  limiting  value  of  the  variance  of  the  sum  of 

k adjacent  to  X^'s,  standardized  by  its  value  for  independent 
exponential  variates; 
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(2.21) 


J 


lim 
k “ 


var  (X) 
{E(X)}^ 


1 + 2 I 


= 1 + 2 I p.  , 
j=l  3 


and  is  proportional  to  the  initial  point  of  the  spectrum 
of  the  process  {x^}.  For  the  EAR(2)  process  this  is, 
using  (2.17) , 


J 


2YiZi  2y2Z 

^ * Tr:iqT  * 


(2.22) 


This  becomes  very  large  as  the  roots  and  22  approach 

1,  indicating  that  the  process  has  very  long  term  dependence 
in  it. 

Some  other  properties  of  the  EAR (2)  process  which  are 
of  interest  are  that  the  regression  of  on  and 

is  linear  in  the  given  values  **^<5  ^i-2  *i-l 

X 

i-2 

E(Xi|Xi_i  = x._3^;  X^_2  = x^_2) 

= (l-a2)otj^x^  2 ^ A (l-Oj)  (l”Oj^+<*2^  ' (2.23) 


and  that  the  conditional  correlations  of  and  given 

X.  ,,  ...  , X,  » w are  zero  for  A * 3,4,...  . 


I 

I 


We  note  too  that  the  EAR (2)  model,  like  the  EAR(l)  model. 


is  slightly  degenerate  in  that  one  obtains  runs  of  X^^'s  which 


are  fixed  multiples  of  the  previous  or  • 

Joint  distributions,  higher-order  joint  moments  and  partial 


sums  of  the  {X^}  process  will  be  considered  elsewhere, 


3.  THE  EXPONENTIAL  MOVING  AVERAGE  EMA(q)  MODEL 


The  EAR(l)  model  of  Gaver  and  Lewis  (1975-78)  led  to  the 


development  by  Lawrance  and  Lewis  (1977)  of  a corresponding  first- 


order  exponential  moving  average  model;  this  took  the  form,  in 


the  backward  case,  of 


w.p.  6 


X,  = 


^ I 3E^  + E^_j^  w.p.  (1-6), 


(0  <6<  1;  i = 0,+l,+2, . . .)  (3.1) 


where  the  {E^}  is  again  a sequence  of  i.i.d.  exponential (X ) 
variables . The  X^ ' s have  an  exponential  marginal  distribution 


and  are  only  serially  dependent  for  lag  one;  this  model  is  highly 


tractable  and  a full  account  of  the  statistically  useful  properties 


was  obtained.  The  forward  model  is  defined  as  a random  mixture 


of  6E^  and  6Ej^  + , instead  of  BEj^  and  BE^^  + 


Lawrance  and  Lewis  (1977)  pointed  out  briefly  that  exten- 


sions to  second-order  moving-average  models  are  possible;  thus  we 


replace  E^_j^  in  (3.1)  by  another  EMA(l)  variable,  a random 
linear  combination  of  ®l^i-l  ^i-2'  will  still 


be  exponentially  distributed  and  independent  of  the  E.  variable. 
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Thus  the  second  order  backward  model,  EMA(2),  becomes 


62^1 


®2=i  * (1-62)61 

®2®i  * ®i-2  '^-P-  (1-62)  (1-e^)  , 


w.p.  6- 


where  0 £ 6^,  62  £ i * ...  . The  serial  dependency 

of  this  model  clearly  stops  at  the  second  lag  and 


Pi  = (l-Bj)  6^{1  - (1-32)  6j^};  P2  = $2(1  - 3^)  (1  - 32).  (3. 


This  model  reduces  to  the  independence  case  and  the  EMA(l)  model 
for  various  values  of  3j^  and  32*  The  general  EMA(q)  model  takes 
the  form 


3 E. 

q 1 

3 E.  + B tE.  . 
q 1 q-l  i“l 


w.p.  b 


w.p.  b 


*i  = 


3E.+B  ,E.  ,+•••+  B,E. 

^q  1 q-1  1-1  ^1  i-q+1 


w.p.  b. 


Vi  * Vl'^i-l  * •••  * »l®i-q+l  * '^i-q  "-P- 


for  0 < 6j.  Bj-  •••  • 6 < Ij  1=0,  +1,  +2,  .. 


. and 


i = q+1  , 


(l-3_)  •••  (1-3.. ) 3-.  , q > i > 2, 


Note  that  the  can  be  obtained  uniquely  from  the  b^'s;  there 

are  q+1  b^'s,  but  only  q 8's,  since  the  sum  of  the  b^'s  is  equal 

to  one.  It  is  simple  to  see  that  the  {X^}  have  exponential 
marginal  distributions.  The  serial  correlations  for  this  model 
clearly  have  the  cut-off  property  associated  with  moving  average 
schemes;  they  can  be  obtained  without  recourse  to  difference 
equations.  Premultiplication  of  (3.4)  by  (r  ^ 1)  and  then 

taking  an  expectation  gives 


E(XjX^._^)  . + ...  . b^)  E(E.X^.^) 


* * •••  * ‘’l' 


+ • 


+ 6i(b2  + bj)  E(E^.g^^X..^)+b^E(E..^X..^).  (3.6) 


This  simplifies  since  there  are  the  relations 


+ •••  + bi)  = b.^j 


(q  > i > 1)  . 


Thus  on  converting  (3.6)  to  covariances  we  have 


Cov(X 


Vl-m  ' 


(3.7) 


(3.8) 


The  covariances  on  the  right-hand  side  of  (3.8)  follow  from  (3.4)  as 


Cov(E.  ,X. ) = 
1— m 1 


Vm+l  Oin-iq 

0 otherwise. 


(3.9) 


Thus  (3.8)  becomes 


Cov(X^,X^_j.)  = ? b 


Vm+r+1 


) (l<r£q)  (3.10) 


with 


= Corr(X^,X^_j,)  = 


q-r+1 
y b b 

V v+r 

v=l 


(1  < r < q) 


(q+1  £ r < ««) 


(3.11) 


Thus  the  serial  correlations  are  just  lagged  products  of  the  b^ 
sequence  and  the  formula  (3.11)  is  completely  analogous  to  the 
formula  for  the  serial  correlations  of  the  standard  MA(q)  process; 
see  Box  and  Jenkins,  1970,  ?.  68.  It  can  be  seen  from  (3.11)  that 
all  the  correlations  are  nonnegative  and  it  may  further  be  shown 
that  they  are  bounded  above  by  1/4.  Note  too  that  since  the  's 

are  lagged  products  of  the  b^  sequence,  it  is  not  possible  to 
determine  the  b^'s  uniquely  from  the  Therefore  it  is 

not  possible  to  uniquely  determine  the  from  the  p^*^^'s. 

Consider  now  the  index  of  dispersion  J,  defined  at  (2.21)  . 
It  can  easily  be  shown,  from  (3.11),  that  for  the  EMA(q)  process. 


this  is  given  by 


J = 2 - J bj 
P = 1 


(3.12) 


This  is  maximized  when  all  b^s  are  equal  and  thus  3^^=  1/(1+*.), 
Z = l,2,...,q.  These  values  of  3^;^  have  the  property  that  they 
give  equal  weights  to  the  q+1  possible  linear  combinations  which 
can  make  up  an  X^,  that  is  b^  * l/(l+q),  i = 1,2,...,  q+1. 


i 

i 


F' 


► 


The  maximum  values  of  J are  then,  as  q increases,  1.5,  1.666, 
1.750,  1.8000  and  generally,  1 + q/(l+q)  with  2 as  limiting  value; 
thus  beyond  a certain  point,  increasing  the  order  of  the  moving 
average  (which  can  conceptually  go  to  infinity),  has  little  effect. 
This  implies  that  the  over  all  dependence  in  the  process,  as 
expressed  by  J,  is  bounded  and  that  very  high  values  of  q do 
not  substantially  increase  dependence. 

A convenient  notation  for  the  EMA(q)  sequence  {X^}  is 
, meaning  that  has  a moving  average  structure  of  order  q 

over  E^,  ...  , using  the  parameters  3^,  5q_i»  •••/ 

In  this  notation,  can  be  expressed  in  terms  of  by 

the  recursion 


= 3 E.  + I.' 
1 q 1 1 


i-l 


q — 1,2, 


(3.13) 


where  is  an  independent  sequence  of  binary  variables 

taking  value  zero  with  probability  = ^q* 


4.  THE  EXPONENTIAL  AUTOREGRESSIVE -MOVING  AVERAGE  PROCESSES 
EARMA(p,q) . 

4a . Definitions 

We  have  defined  both  autoregressive  processes  and  moving- 
average  processes  in  exponential  variables  of  any  specified  orders, 
p and  q.  Here  we  bring  them  together  into  a single  process. 
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EARMA(p,q),  although  it  will  be  seen  that  the  method  of  combination 
is  not  unique.  We  will  then  have  a process  of  great  flexibility 
in  modelling  dependent  exponential  variables,  bearing  favorable 
comparison  with  the  standard  ARMA(p,q)  process  in  modelling  dependent 
Gaussian  variables,  Jacobs  and  Lewis  (1977)  linked  the  two  first 
order  exponential  processes  EMA(l)  and  EAR(l),  giving  an  EARMA(1,1) 
mixed  model  and  obtained  the  serial  correlations,  some  higher 
order  explicit  results  and  discussed  central  limit  and  mixing 

V 

properties.  For  the  general  mixed  process  we  shall  give  two  types 
of  model  but  restrict  ourselves  to  their  correlational  properties, 
and  in  particular  derive  the  difference  equations  satisfied  by 
the  serial  correlations;  these  are  similar  but  not  identical  to 
those  of  the  standard  ARMA  process.  The  special  process  EARMA(2,1) 
and  EARMA(1,2)  will  be  considered  in  more  detail. 

In  seeking  exponentially  distributed  mixed  autoregressive- 
moving  average  processes  we  will  work  from  the  pure  (backward) 

moving  average  process  EMA(q)  given  in  (3.4).  One  reason  why 
the  exponential  moving  average  and  exponential  autoregressive 
models  are  appealing  and  tractable  is  that  they  are  expressed 
in  terms  of  independent  exponential  variables.  If  this  property 
is  to  be  carried  over  into  the  mixed  models,  then  the  autoregressive 
contribution  should  enter  without  violating  this  feature;  thus, 
to  construct  the  EARMA(p,l)  process  we  replace  the  variable 

in  the  EMA(l)  of  (3.1)  by  an  EAR(p)  variable.  This  is 
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independent  of  in  (3.1)  because  it  is  a function  only  of 

E.  ,,  E.  ...  . The  defining  equation  for  the  EARMA(p,l)  process 
is  thus 

IBE^  w.p.  B 

(0<6£1;  i=0,+l,+2,  . . .)  (4.1) 

BEi  + w.p.  1-B 

which  is  the  model  treated  by  Jacobs  and  Lewis  (1977)  when 
p = 1.  Similarly,  (3.4)  leads,  on  replacing  E^_g  by  a|^ 
to  the  EARMA(p,q)  process,  with  equation 

"•p-  Vi 

w.p.  bq 

( 

w.p.  b2 
w.p.  bj^ 

for  i = 0,  +1,  +2,...,  where  the  b^'s  are  defined  at  (3.5)  and 
0 £ B^/  •••  t £ 1 . Writing  as  a variable  in  the 

EARMA(p,q)  process  based  on  the  moving  average  parameters 
B , •••  • mixed  process  can  be  defined  recursively  as 

w.p.  6 

q 

(i=0 , +1 , +2 , . . . ) 


/BE. 

I ' 

^ ) 6 E.  + 


B E. 

q 1 

^q^i  ^ ^q-l®i-l 


X.  = 
1 


Vi  * VA-l  8A-q*l 

Vi  * VA-1  8l®i-qrt  * ^'i^q 


W.p.  1-e 


(4.3) 


This  class  of  models  will  sometimes  be  written  as  EARMA  {p,q) 
to  signify  that  it  is  based  on  a backward  moving  average. 

Consider  further  the  structure  of  tJ^e  mixed  model;  for  instance, 
depends  on  E^,  •••  and  not  on  E^^2'  •••  • 

paralleling  the  standard  model  ARMA(p,q)  model.  In  contrast 
to  the  standard  model  it  is  also  possible  that  the  autoregressive 
aspect  could  be  absent  in  stretches  of  the  process  when  one  of 
the  pure  moving  average  selections  is  chosen  each  time.  Dependency 
would  still  be  retained  in  the  model  by  the  moving  average  part 
(apart  from  the  q = 1 case  of  course) ; while  this  is  not  at 
all  unnatural  there  can  be  other  situations  when  it  is  desired 
to  always  have  autoregressive  dependency.  Such  considerations 
lead  to  alternative  mixed  models;  initial  concern  at  the  non- 
uniqueness of  these  models  is  best  allayed  by  realizing  that 
there  is  nothing  unique  about  the  standard  Gaussian  mixed 
ARMA(p,q)  models.  In  an  alternative  formulation  of  the  general 
mixed  model,  to  be  denoted  by  EARMA^(p,q),  the  shifted  form  of 
the  forward  moving  average,  briefly  mentioned  in  Section  3 is 
used.  The  retention  of  independence  between  successive  terms 
in  the  model  then  leads  to  the  EARMA^(p,l)  process  as 

w.p.  , 


.1 


6 A 

*^1  i-1 


X. 


( r\\ 


(i  = 0,+l,+2,  . . .) 


(4.4) 


and  to  the  EARMA  (P/q)  as 


= 


q i-q 

B A.^P^  + e ,E, 
q i-q  q-1  i-q+1 


Vi-q  * VlVq+1  ^•••■^ 

'Vl-q  * VA-q+1  *•••■" 


w.p.  b 


q+1' 


w.p.  bg. 


w.p.  b,. 


w.p.  b, , 


(4.5) 


for  i = 0,  +1,  +2,  ...  . It  can  be  seen  from  (4.5)  that  this 
has  the  structure 


(p^q) 


's  a!p> 

q l-q 


w.p.  B, 


B AfP^  + w.p.  l-6_ 

q i-q  1 q 


( i=0 , +1 , +2 , . . .) 


(4.6) 


where  is  a variable  in  the  shifted  forward  moving  average 

model  of  order  q-1  using  the  parameters  ^q-i'  ' ^1*  Thus 

the  autoregressive  dependence  is  always  present,  and  is  lagged 
q values  in  arrears;  the  moving  average  variable  gives  greater 
flexibility  to  the  initial  form  of  the  dependence,  and  differs  for 
the  two  models.  In  EARMA~(p,q)  the  most  recent  is  always 

included;  then  with  probability  ^q-1  ® linear  combination 

of  E^  and  is  included,  and  so  on  moving  back;  thus 

because  of  the  certain  addition  of  a new  E^  each  time  there 
cannot  be  runs  of  scaled  values;  further,  this  is  a back  progression, 
natural  in  many  cases . The  price  of  these  features  is  that  the 
model  can  exhibit  patches  of  independence  when  only  the  E.  is 
chosen,  i.e.  the  autoregressive  tail  is  not  chosen.  This  cannot 


I 


happen  in  the  EARMA^(p,q)  where  the  autoregressive  dependency 

is  always  present;  however,  complicated  but  weak  runs  of  scaled 

values  are  just  possitle  in  the  EARMA^(p,q)  model,  arising  from 

a low  order  autoregressive  contribution  sucessively  being  chosen 

on  its  own.  Such  a situation  would  be  extremely  rare.  Our  general 

feeling  is  that  in  practice  there  would  not  be  much  to  choose 

from  between  the  two  types;  a third  type,  more  similar  to 

EARMA^(p,q)  than  the  other  can  be  formed  by  interchanging  the 

processes  in  (4.6)  with  a suitable  shifting  of  scale.  This  is  : 

not  considered  here. 

j 

4b . Correlations  for  the  backward  mixed  model  EARMA(p,q) 

We  next  derive  equations  satisfied  by  the  serial 
correlations  of  the  EARMA  (p,q)  process,  denoted  here  simply  as 
EARMA(p,q).  Multiplying  each  side  of  the  defining  equations 
(4.2)  by  ^ 0)  taking  expectations,  gives 


k 


+1 , +2 , . . . 


(4.7) 

This  equation  is  not  valid  for  r » 0 since  the  expectation  of 
the  mixture  is  not  the  mixture  of  the  expectations  when  the 

23 


ElXiXi.r)  = + ...  + bj)  ECEjX^.^) 

* '=1> 


+ ...  + Bi(bj  + bj) 


variables  are  identical.  Following  equations  (3.6),  (3.7)  and 
(3.8),  the  covariance  form  of  (4.7)  becomes 


Cov(X^,Xj^_^) 


X Vl-»  + ■’1  ' 


(4.8) 


It  now  becomes  easier  to  worlc  mainly  in  terms  of  correlations  and 
to  define 


a Cov(X^,X^_j.)  and  * Corr  (E^,X^_^) 


(4.9) 


for  r = 0,  +1,  +2,  ...  . Since  the  and  X^  variables  have 
the  same  marginal  exponential  distribution,  (4.8)  becomes 


P = r.K  m Corr(AjPi,  X.  _) 

q+l-m  r-m  1 i-q  i-r 


for  r = +1,  +2,  ...  . To  calculate  the  cross-covariances 
between  the  autoregressive  and  mixed  process,  we  first  note 


(4.10) 


that 


“I'^i-l-q 

a A^P^ 
“2^i-2-q 


a aIP^ 
p i-p-q 


w.p.  a 


w.p.  a. 


w.p.  a^ 


+ £.*  , i = 0,+l,+2,  ... 
i-q  - - 


(4.11) 


% \ r ^ . 

V: 


I 

I 


'I 

I 

1 1 


following  the  notation  at  (2.5)  and  (2.6).  The  c^-term  in  (4.11) 
has  a distribution  which  depends  only  on  such  as  was  determined 

for  p « 2 in  Section  2.  Multiplication  of  (4.11)  by  in 

order  to  calculate  correlations  leads  to 


Corr(A!Pq,X^_^) 

“ Jl  Cov(e._q,X._^)/Var(E)  , 


r = 0 , +1 ,+2 , ...  . 


(4.12) 


We  now  wish  to  substitute  from  (4.10)  for  the  correlations  in  (4.12) 
and  so  obtain  a difference  equation  for  p^.  However  we  do  not 
have  (4.10)  in  the  case  r = 0.  Thus  in  (4.12)  when  i-r- (i--^--q)  =q 
this  substitution  is  not  possible,  that  is  when  I = r if 
r < D.  In  this  case 


Corr (AfP^,X^)  * bj^  , 


(4.13) 


as  may  be  seen  from  (4.2).  Thus  (4.10)  and  (4.12)  lead  to 


“ Jq  *'q+l-m*r-m 


+ bj^Cov(G^_g,X^_j.)/Var  (E) 


(4.14) 
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2 

for  r = if  r > p then  the  term  is  omitted. 
The  asterisk  denotes  that  the  i » r term  is  omitted  from  this 
summation  when  r £ p.  The  equation  simplifies  on  defining 


P 


and 


Cr-q  “ Cov(e._q,X._j.)/Var(E)  ; 


(4.15) 


equation  (4.14)  may  then  be  written 

Wr-i  * T 


r-m” 


(4.16) 


This  is  the  desired  general  result. 

Noting  that  = 0 for  j _>  1,  we  see  that  for  r p+qr 
(4.16)  reduces  to  an  rth  order  difference  equation 


Pr  = “i^iPr-l  “2^2Pr-2  + ••*  + “pVr-p  ^ 

which  is  the  same  as  (2.16)  for  the  EAR(p)  process.  To  calculate 
the  initial  p+q-l  serial  correlations,  •••#Pp+q_i'  need 

Kg,  , ...  , ^0*  ^“1^  • • • » q+1  * explicit 

expressions  for  these  quantities  are  given  in  the  Appendix.  The 
correlation  structure  of  EARMA(p,q)  processes  is  thus  similar  to 
that  of  the  standard  ARMA(p,q)  processes;  the  only  difference  is 
that  initial  calculation  of  the  first  p+q  serial  correlations 
are  needed  to  start  the  difference  equation  (4.17),  rather  than 
the  first  p as  in  the  ARMA(p,q)  case.  Note  finally  that  (4.16) 
is  only  strictly  true  when  p 1,  q £ 1,  although  similarities 
with  the  equations  for  the  (p  » 0,  q)  case  and  (p,  q = 0)  case 
are  apparent. 
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These  results  all  apply  to  the  backward  model*  EARMA(p,q); 
for  the  forward  EARMA^(p,q)  model  (see  (4.5))  slightly 
different  correlation  equations  apply.  The  main  difference  is 
that  (4.8)  is  now 


Corr  (X^,X^_^)  = Corr  (A 


i?q'*i-r^  _L  ^q+l-m*^r-q+m  * 


m*=l 


(4.18) 


Using  (4.12)  we  then  find,  corresponding  to  (4.16),  that 
^r  ^q+l-m^^'r-q+m  " ^r-i-q+m^  ^q+l^r- 


q 

(4.19) 


for  r = 1,2, 


of  Kq,  K_^, 


with 
, K 


as  before  explicit  calculation 


-p-q+2  =0'  =-!■ 


--q+1 


are 


required  to  obtain  the  first  p+q  serial  conditions.  Further 
comparisons  between  the  two  types  of  model  will  be  dealt  with 
elsewhere . 


5.  SPECIAL  CASES  OF  THE  EARMA(p,q)  PROCESS 

We  shall  give  the  explicit  versions  of  the  correlation 
equations  (4.16)  in  the  cases  (p  > 1,  q >=  1)  , (p  =*  2,  q = 1) 
and  (p  s 1,  q = 2);  these  are  considered  to  be  potentially  the 
most  useful. 
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(i)  EARMA(1,1) 


This  model  will  be  written  in  the  notation 


X.  = 
1 


6E^ 


w.p.  S 
j 6E^  + w.p.  1-8 


where 


^i  “ P^i-l  + ^i' 


e . = 
1 


° w.p.  P 
w.p.  l-p 


The  correlation  equations,  from  (4.16),  are  then 


Pi  ^2^1  ^1^0  ' ^r  “ “l^l^r-1  ' (r  ^ 2) 


giving 


= p(l-e)^+  (l-p)  6(1-B),  p^  = PP^_j^,  Ir  > 2)  . 

This  agrees  with  the  result  (2.4)  of  Jacobs  and  Lewis  (1977). 

(ii)  EARMA(2,1) 

Using  (4.16)  to  obtain  the  results  for  p^  and  p^  we  have 
2 

Pi  » - K ,}  + b,C«  ; 


with 


(iii)  EARMA(1,2) 


6 . FURTHER  DEVELOPMENTS 


There  are  many  facets  and  properties  of  the  EARMA(p,q) 
process  which  will  be  investigated  in  later  papers.  Some  of  these 
properties  have  been  investigated  for  the  EAR(l)  process,  the 
EMA(l)  process  and  the  EARMA(1,1)  process  by  Gaver  and  Lewis 
(1975-78),  Lawrance  and  Lewis  (1977)  and  Jacobs  and  Lewis  (1977) 
respectively.  They  include  mixing  conditions,  infinite  divisibility, 
stationary  initial  conditions,  joint  distributions,  distributions 
of  sums,  spectra  and  higher  order  correlations.  It  should  also 
be  emphasized  that  all  the  serial  correlations  in  the  EARMA(p,q) 
process  are  positive;  this  aspect  of  the  process  can  be  broadened 
by  considering  antithetic  processes  and  will  be  discussed  else- 
where . 

Another  important  question  which  arises  is  whether  there 
are  non-normal  distributions  other  than  the  exponential  for  which 
mixed  autoregressive,  moving  average  structures  can  be  defined 
analogous  to  EARMA(p,q).  The  general  question  is  difficult  and 
has  been  considered  by  Gaver  and  Lewis  (1975-78)  for  the  first 
order-autoregressive  process.  However  it  is  clear  that  by  adding 
two  independent  EARMA(p,q)  processes,  say  } and 

we  obtain  a process  which  has  Gamma (2)  marginal  distributions, 

P{X.  £ xf^^  + xf^^  < x}  = / V e"'^  dv  , 

^ ^ ^ 0 
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and  ARMA(p,q)  correlation  structure.  Some  analysis  shows  that 
this  process  is  generated  directly  as  a mixture  over  two  independent 
i.i.d.  exponential (X)  sequences,  and  possibly 

scaled.  It  would  be  interesting  to  extend  this  process  to  the 
fractional  Gamma  case,  i.e.  < not  an  integer  but  it  is  not 
clear  whether  this  process  exists  or  how  to  construct  it. 

For  the  first-order  autoregressive  case,  it  is  simple  to 
show  that  the  random  variable  defined  at  (2.12)  is  infinitely 

divisible  and  therefore  that  the  solution  of  (2.4)  for  exists 

when  is  Gamma  (ic,X)  and  has  a Laplace-Stiel  jes  transform 

which  is  just  (2.11)  raised  to  the  power  ic. 


E(e  ^)  = (p  + (1-p) 


X 


IC  > 0,  X > 0 (6.1) 


However  it  is  difficult  to  invert  this  transform,  or  to  generate 
the  random  variable  on  a computer  unless  < is  an  integer. 

This  should  give  an  indication  of  some  of  the  interesting 


theoretical  questions  which  are  raised  by  the  EARHA  process. 


K 

K- 


1. 


APPENDIX:  Calculations  of  ^ ” 0,1,2,  ...}  and 

{C_^,  r = ,l,...,q-l}  for  EARMA(p,q)  models 


By  definition 


K_^  = Corr(E^,X^^^)  = Corr(E^_^,X^)  . 


(A.l) 


By  the  usual  process  of  multiplication  and  expectation  it  is  found 
that 


*^q-r+l  0 < r < q-1  , 


Cov(E,  „,X, ) = 


i ■ I Cov(E^.^,A<P 


(P) 

q 


) q < r < 


(A. 2) 


and  in  terms  of  correlations 


q-r+1 


0 < r < q-1  , 


K 


-r 


(A. 3) 


Corr  (E^_^,AfPq)  q £ r < «■  . 


The  calculation  of  the  cross-correlations  between  the  independent 
exponential  sequence  and  the  derived  autoregressive  sequence 
proceeds  in  the  usual  way,  and  gives 


Corr (E^,A^P^ ) = Cov (E^, e^) /Var (E)  = d^  ; 


min ( j ,p) 


(A. 4) 


Corr(E.  .,AfPS  = I at. a.  Corr  (E . . ,A fP j ) . (A. 5) 

1“]  1 *•  *'  J-  J 1“X, 
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I 

I 


We  only  need  (A. 5)  when  P >.  2 and  for  j » l,2,...,p-2, 
Recursively  then  (A. 4)  and  (A. 5)  give 


Corr(E^_j,AfP^ ) 


= (aia^)do 


= {(a^a^^)^  + (0232)  }dQ 


= {(a^aj^)'^  + 2(aj^aj^)  (“2^2^  *^0 


(j=l) 


(j=2) 


(j=3)  (A. 6) 


4 2 

= { (Oj^a^)  + 3(aj^aj^)  (“2^2^  2 (Oj^aj^)  (a^a^) 


+ (0232)”^  + (a^3^)}dQ 


(j=4) 


Further  expressions  will  be  evident  based  on  the  fact  that  the 
sum  of  the  suffices  equals  j,  that  all  possible  such  groups 
of  terms  are  present,  and  that  the  coefficient  of  a particular 
term  is  the  number  of  distinct  orders  of  a term  of  that  type. 


By  definition  we  have 


C_^  = Cov(e^,X^^^)/Var(X)  = Cov  (Ej^_^,X^)/Var  (X)  . 


(A. 7) 


AS  at  (A. 2)  above  we  have 


Cov(ei_r,Xi)  = bq+i_r  <^i-r'®i-r^  ' 0 < r £ q-1. 


(A. 8) 


Hence 
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